Properties of Associative Memory Neural Networks concerning Biological Information Encoding

Associative abilities of the neural networks concerning the information coding in the real neural systems are studied. Two models which we adopted are the sparsely coded neural network and the oscillator neural network. We theoretically analyze such models with the replica theory and the theory of the statistical neurodynamics. These theories enable us to describe the states of the systems which consist of a number of units with a few macroscopic order parameters. It is well known that a sparsely coded network in which the activity level is extremely low has intriguing equilibrium properties. Hence, first, we study the dynamical properties of a neural network designed to store sparsely coded sequential patterns rather than static ones. Applying the theory of statistical neurodynamics, we derive the dynamical equations governing the retrieval process which are described by some macroscopic order parameters such as the overlap. It is found that our theory provides good predictions for the storage capacity and the basin of attraction obtained through numerical simulations. The results indicate that the nature of the basin of attraction depends on the methods of activity control employed. Furthermore, it is found that robustness against random synaptic dilution slightly deteriorates with the degree of sparseness. Second, we study the static and dynamical associative abilities of an oscillator neural network in which information is encoded by the relative timing of neuronal firing. In order to analyze such abilities, we apply the replica theory and the theory of statistical neurodynamics to the oscillator model. Using the theoretical results from these analyses, we can present the phase diagram showing both the basin of attraction and the equilibrium overlap in the retrieval state. Our results are supported by numerical simulation. Consequently, it is found that both the attractor and the basin are preserved even though dilution is promoted. Moreover, as compared with the basin of attraction iii in the traditional binary model, it is suggested that the oscillator model is more robust against the synaptic dilution. Taking it into account the fact that oscillator networks contain more detailed information than binary networks, the obtained results constitute significant support for the plausibility of temporal coding.